To understand how best to use observations of Type Ia supernovae (SNe Ia) to obtain precise and accurate distances, we investigate the relations between spectra of SNe Ia and their intrinsic colors. Using a sample of 1630 optical spectra of 255 SNe, based primarily on data from the CfA Supernova Program, we examine how the velocity evolution and line strengths of Si II 6355 and Ca II H&K are related to the B-V color at peak brightness. We find that the maximum-light velocity of Si II 6355 and Ca II H&K and the maximum-light pseudo-equivalent width of Si II 6355 are correlated with intrinsic color, with intrinsic color having a linear relation with the Si II 6355 measurements. Ca II H&K does not have a linear relation with intrinsic color, but lower-velocity SNe tend to be intrinsically bluer. Combining the spectroscopic measurements does not improve intrinsic color inference. The intrinsic color scatter is larger for higher-velocity SNe Ia --- even after removing a linear trend with velocity --- indicating that lower-velocity SNe Ia are more "standard crayons." Employing information derived from SN Ia spectra has the potential to improve the measurements of extragalactic distances and the cosmological properties inferred from them.